Most modern web services implement some kind of rate limiting to mitigate the effect of online attacks. Generally, rate limiting systems block IP addresses that exceed some threshold of activity. This technique is used to protect against denial of service attacks (“DOS”) and other types of malicious activity based on bombarding a server with requests or other types of activity (e.g., web scraping, online password attacks, etc.).
Conventional rate limiting breaks down when receiving communications from large Network Address Translation (“NAT”) systems, where a large number of users appear to the web service as a single IP address. NAT is a methodology of modifying network address information in IP packet headers while they are in transit across a traffic routing device, for the purpose of remapping one IP address space into another. NAT is often used to hide an entire IP address space, usually consisting of private network IP addresses, behind a single IP address in a public address space. This mechanism can be implemented in a router that uses translation tables to map the hidden addresses into a single, visible IP address, and readdresses the outgoing IP packets on exit so they appear to originate from the routing device. In the reverse communications path, responses are mapped back to the originating IP addresses using the translation tables.
Without a reliable way to identify NAT systems and how many users are behind them, it is difficult for a web service to set appropriate rate limiting thresholds. If the threshold is set too high for the actual number of users behind the NAT, denial of service attacks and the like are not deterred. On the other hand, if the threshold is set too low, legitimate users will be blocked from using the service.
It would be desirable to address these issues.